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Abstract 

We consider the distribution P{(j)) of the Hatano-Sasa entropy, (p, in reversible and irreversible 
processes, finding that the Crooks relation for the ratio of the pdf 's of the forward and backward 
processes, Pf{4>)/Pr{—4') = e*^) is satisfied not only for reversible, but also for irreversible processes, 
in general, in the adiabatic limit of "slow processes." Focusing on systems with a finite set of discrete 
states (and no absorbing states), we observe that two-state systems always fulfill detailed balance, 
and obey Crooks relation. We also identify a wide class of systems, with more than two states, that 
can be "coarse-grained" into two-state systems and obey Crooks relation despite their irreversibility 
and violation of detailed balance. We verify these results in selected cases numerically. 

PACS numbers: 05.40.-a,05.70.Ln,05.20.-y 
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I. INTRODUCTION 



In recent years much effort has been devoted to the characterization of nonequihbrium 
systems for which no general theoretical framework is available. Some notable progress has 
been achieved in the study of rather generic properties of these systems, as for example the 
entropy production and currents in nonequihbrium steady states iHg] , the aging phenomena 
in systems relaxing towards steady states 71], or the fluctuation properties of systems driven 
out of a steady state j8l4l4|. Interestingly, a large part of this progress is closely related 
to the discovery of various fluctuation and work theorems that provide generic statements 
applicable to large classes of systems. 

One of the best known theorems is Crooks relation for a system, initially in equilibrium, 
that is driven out of equilibrium through a time-dependent process. Repetition of this 
forward process allows to determine the probability distribution Pp{Wd) of the dissipative 
work Wd- When comparing this probability distribution with that obtained from the time- 
reversed process, called Pr, one observes the following very simple relation 

PF{md)/PRi-/3Wd) = e^^-* , 

with P = l/iksT). This detailed fluctuation theorem is closely connected to other relations, 
notably Jarzynski's work theorem. The latter is an integral fluctuation relation that relates 
for the special case of equilibrium initial and final states the free energy difference AF to 
an average over all processes leading from one state to the other: 



-/3AF 



ler situ- 



where W is the work done on the system. 

Both the Jarzynski and the Crooks relations have been generalized to various ot 
ations. Assuming microreversibility, integral and detailed fluctuation theorems |l2, 
have been derived for the Hatano-Sasa entropy 0, also called the driving entropy production, 
a quantity that is closely related to the excess heat. For an equilibrium system reduces 
to the dissipative work Wd and the theorems become identical to the Jarzynski and Crooks 
relations. In addition, an integral fluctuation theorem has also been formulated for the 
housekeeping heat ISj. Finally, it was shown recently that under the same assumptions of 
microreversibility integral and detailed fluctuation theorems also hold for the adiabatic and 
nonadiabatic trajectory entropies |l6| . 



In absence of microscopic reversibility, as it is for example the case in reaction-diffusion 



systems, a detailed fluctuation t 



reversed path, does not hold 



Il7|. 



leorem, which is based on a forward path and the time- 



However, recent studies of two of us revealed that for 



the Hatano-Sasa entropy the deviations of the fluctuation ratio from a simp 
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e exponential 



2l|. 



contain non-trivial information on the trajectories in configuration space 

In this work we revisit the fluctuation relations involving the Hatano-Sasa entropy when 
driving a system out of a steady state. We confirm that an integral fluctuation theorem is 
always recovered, irrespective of whether the processes are reversible or not. In the limit of 
slow processes, where the time between changes of some external parameter is long enough 
for the system to reach the steady state, a detailed fluctuation theorem prevails. Finally, 
we discuss many-state processes that are irreversible and violate detailed balance and show 
that for large classes of these processes Crooks relation for the Hatano-Sasa entropy remains 
valid. This remarkable result is obtained through a coarsening theorem that states that 
some many-state processes can be coarse-grained into systems with fewer states that have 
the same spectrum of and the same probabilities for each as the original systems. 

Our paper is organized as follows. Sections II and HI focus on exact results on the realm 
of validity of fluctuation relations involving the Hatano-Sasa entropy. Section II is mainly 
devoted to the case of slow processes, whereas in Section HI we discuss the coarse-graining 
procedure and its consequences for the detailed fluctuation theorem in many-state processes. 
In Section IV we illustrate our main results through numerical studies of selected systems. 
Finally, Section V gives our conclusions. 



II. SLOW PROCESSES 



process driven by an external pa- 



22l |. The system is taken through a 



We consider in the following a stochastic dynamica, 
rameter 7, and characterized by its internal state x{t) 2 
series of changes, over a total period of time, T, whereupon the parameter 7 is varied from 
7 = 7o to 7 = 7Af in a series of M + 1 steps, each lasting a time r = T/ (M + 1). We denote 
the steady state pdf of the system, under 7 = 7^, by p{x, 7^), and write it, more compactly, 
as 

p(x,7„) = pn{x) . 

Let Pn{x'\x") denote the transition probability, in the time interval r, from the initial 
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state x{t = 0) = x" to the final state x{t = r) = x', when the driving parameter is 7 = 7„. 
The weight of the particular path [x, 7] = (xq, 70), (a^i, 7i), • • • , {xm, 7m) is given by 

M-l 

Wdx,^]) = PoiXo) Yl Pn+liXn+l\Xn) , (l) 
n=0 

and the average of a dynamical quantity, over all possible histories, is given by 

(Q) = J " J ^( 7]) dxo dxi - ■■ dxM ■ 

Suppose that the initial pdf of the system, at time t, is p{x,t) = Pn{x), then the pdf at 
time t + r is 

j Pn{x\y)pn{y) dy = Pn{x) , (2) 

by the very definition of the steady state: Since the system is in the steady state to begin 
with, it will remain there regardless of how long it evolves (assuming that the external 
parameters are held constant). If the initial state is different, for example, p{x, t) = 6{x — x'), 
the pdf after time r. 



p{x, t + r) = J Pn{x\y)5{y - x') dy = Pnix\x') 



is not necessarily equal to the steady state Pn{x). Note, however, that if r — )• oo the system 
would eventually arrive at the steady state, regardless of its initial condition: 



p{x,t + ^) = j Pn{x\y)5{y - x') dy = P„(a;|x') p„(x) 



as r — )■ cxD . (3) 



We call slow process a process where the system is allowed to reach the steady state after 
each incremental change in the external parameters (by letting r — )■ oo). Using ([3]), we see 
then that the weight of a path for a slow process assumes the much simpler form: 

M 

W^slow(k, 7]) = IJ PniXn) ] sloW prOCCSS. (4) 

n=0 

In the following we explore the validity of different fluctuation relations within the setting 
just described. 

A. The integral fluctuation theorem, (e""^) = 1 

Define A0, during one step, as 

Acpn = lnp„(Xn) - lnpn+l{Xn) , 
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such that the total change throughout the process is 

M-l M-1 



</) = ^ A0„ = ^ [lnp„(x„) - lnp„+i(a;„)] . (5) 

n=0 n=0 

Using the path weight ([1]), we then have 

f f (x ) 

[e"*") = ■■■ Po{xo) Yl — "^Pn+iixn+ilxn) dxodxi ■ --dxM = 1 ■ (6) 

J J Pn[Xn) 

This can be seen most easily by integrating over the variables Xo,Xi, . . . , xm in this precise 
order, and using the relation Thus, (exp(— 0)) = 1. 

It has to be noted that this result is valid for all processes, as we did not make any 
restricting assumptions. Especially, the processes do not need to be reversible nor slow. 



B. The pdf and the ratio Pf(0)/^'r(-0) 
The pdf of (j) is given by 
P{4') ^ J "' J ^ (^^^ ^[lnp„(x„) -lnp„+i(x„)]^ W{[x,'j])dxodxi---dxM- (7) 

Since the integrals involved are often hard to evaluate, we prefer working with the Fourier 
transform: 

/oo 
P{(l)) e-'^^ d(l) = {e-"^'') . (8) 
-oo 

We now turn to the ratio of the pdf 's for the forward and backward processes. The forward 
process is the process we have been considering all along. The backward process, is the same 
process but where the order of states is reversed. It can be thought of as a forward process 
along the reversed path [x,7] = {xm,1m), {xm-i,1m~i), ■ ■ ■ , {xo,'yo), obtained by making 
the substitution n ^ M — n. Note that in the backward process {xm-, 7a/) is assumed to be 
in the steady state, while (xo,7o) is generically not in the steady state. Thus, the backward 
process is not a perfect time-reversal of the forward process, but the reversal is only in the 
sequence of states. 

Putting ([7]) and ([I]) in ([8]) and carrying out the integration over (j) first, we obtain for the 
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forward process 



M-l 



poi^o) n 



n=0 



po[xo 



A-ik 



M-l 

n 



Pn (-^n) 
Pn+l{Xn, 

Pn i^-^n 



-ik 



Pn+l{Xn+l\Xn) \dXo---dx 



M 



n=l 



'ik 



Pn 



\ik 



dxn ■ ■ - dx 



M-l 1 



_pn[Xn-l)_ 

where, for the second hne, we have rearranged the product and integrated over xm, using 
the fact that / P{y\x) dy = 1. If the states of the system form a discrete set, {A, B,C, . . .}, 
the remaining integrals can be put in the form of a matrices product, 

PoiB)-^'^' 



T [P^(0)] = [pM{Ay\ pM[By\ ...) Fa/-iFm-2 • • • Fi 



V 



(9) 



with 



e'^ can be written as 



Similarly, the transform of Pr(- 



R 



M-\ 



\ 



(10) 



where now 



(Rri 



Pn{X\Y) 



Pn{Y) 



-ik+l 



Upon taking the transpose of the rhs of (ITU]) it is seen that [P/j(— 0)e'^] = J^[Pp{ 
provided that = F„. But R^ = F„ if and only if P„(X|F)p„(F) = P„(F|X)p„(X). This 
last relation is satisfied in all equilibrium processes, where it is known as detailed balance. 

In the limit of slow processes, P{x\y) — )■ p{y), hence P{x\y)p{y) = p{x)p{y) = p{y)p{x) = 
P{y\x)p{x). Therefore R^ = F„ (even for nonequilibrium systems) and the Crooks relation 
Pf{(P) / Pr{-<P) = is satisfied. 



III. COARSE-GRAINING OF MANY-STATE PROCESSES 

As we discuss in the following the validity of Crooks relation is much larger than what 
one would expect naively. In fact, there are large classes of nonequilibrium systems that 
obey this relation, and this despite their irreversibility and the violation of detailed balance. 
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Let us first consider a system with only two states, A and B. Unless there is at least 
one transition (say, from AtoB), the system is trivial and does not evolve at all. However, 
with only A ^ B the steady state of the system is p{A) = and the generahzed entropy 4> 
is not well defined. We therefore need consider only systems that include both transitions, 
A ^ B and B ^ A. Such systems, however, have an equilibrium state that obeys detailed 
balance, and therefore the Crooks relation is satisfied for all (relevant) two-state systems. 

Generally, irreversible processes with more than two states are not expected to obey the 
Crooks relation. We now identify a wide class of such systems that does satisfy the Crooks 
relation, despite their patent violation of detailed balance. These are characterized by the 
following theorem: 

Any system with states ^41, ^42, Si, S2, Ci, C2, , such that all the transition 
rates Ai — >■ Xii are of the form u{Ai, Xii)a, all the rates Bj — > Yji are of the form oj{Bj, Yji)/3, 
the rates Ck — >■ Zk' are uj{Ck, Z^i)^, etc., where all the a;'s are constants and a, ^, 7, . . . are 
driving parameters that are varied arbitrarily through the process' duration, can be "coarse- 
grained" into a system with states A, B,C, . . . , where the "super-states" X = [j-Xi are an 
aggregate of the states Xi. The pdf's P{4>) of the original system and the coarse-grained 
system are exactly the same. 

For the special case where there are only two types of states, Ai and Bj, the system is 
equivalent to a two-state coarse-grained system and the Crooks relation is obeyed! 
To prove the coarsening theorem we write down the rate equations, 

p{Ai) = - J]rate(A ^ Xk)p{Ai) + J]rate(n ^ A)p(n) , 

^k 

p{Bj) = - ^rate(S, ^ Xfc)p(5,) + ^rate(n ^ 5,)p(n) , (H) 

^k Yk 

where p{Xi) now denotes the time-dependent probability for finding the system in state 
Xi and the overdot denotes time differentiation. Using rate(Xi — >■ Yj) — ^u!{Xi — >■ Yj), 
as postulated by the theorem's premise, and writing p{Xi) — ^~^r{Xi) for all the various 
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X-species, the equations become: 

p{B,) = -Y,^{B, ^ X,)r{B,) + Y,^{Y, ^ B,)r{Y,) , (12) 

Xk Yk 

We can now see that in the steady state, when the left side of the rate equations is set to 0, 
the r{Xi) satisfy a homogeneous hnear system of equations with constant coefficients and 
therefore they are constant (independent of the driving fields a,/3, . . .). Then, for states of 
type X, the steady-state probability is 

p{X,) = c{a,(3,...)r'r{X,), (13) 

where c is a normalization constant, determined by p{Xi) = 1. Note that while c is a 
function of the driving fields, it has the very same value for all states (of all types). 
Recall now the expression for 0: 



n 



. n=0 



The important thing is that in each p„(x„)/p„+i(x„)-factor there appears the same state Xn 
(both in the numerator and the denominator), therefore, from f|T3|) . each such term depends 
only on the type of the state x„, but not on which particular one. Thus, for example, 
Pn{Ai) / pn+i{Ai) = CnOin+il Cn+ian is independent of i. It follows that the spectrum of (the 
possible values it attains) depends only on the order that the different types of states are 
met: The paths Xi ^ Yj ^ Zk ^ Wi ^ ■ ■ ■ have the same value of entropy, = 0', for all 
2, j, k,l, . . . The probability for having this particular value of entropy, P{(f)'), is given by 
the sum of all the pertinent path weights: 

W)= Yl Po{X,)Pi{Y,\X,)P2{Zk\Y,)P,{Wi\Zk)--- . 

i,j,k,l,... 

Consider now the coarse-grained system, where we associate all the states of one type 
with a single "super-state," X = [J^ Xj, {X = A, B,C, . . .). The probability that the system 
is initially in super-state X is po{X) = J2iPoi-^i)^ while 

Po{Xi)P,{Y,\Xi)P2iZk\Y^)P,{Wi\Z,) = po{X)P,{Y\X)P2{Z\Y)P,{W\Z) 

i,j,k,l 
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denotes the probability for being at super-state W at step 3, having followed the path 
X ^ Y ^ Z ^ W (and hkewise for additional steps). The conclusion that the coarse- 
grained system has the same spectrum of (f) and the same probabilities for each as the 
original system follows immediately. 



IV. NUMERICAL RESULTS 



In this Section we illustrate our results through the numerical study of some nonequi- 
librium reaction networks. The systems discussed in the following can be in various states, 
called A,B,C, - ■ ■ , and the passage from one state to another takes place with a certain 
reaction rate. 



A. Three states 

Let us start with the simple case of a three-state system with the cyclic reaction scheme 

A H B 

C h A 

The rate equations are given by 

p{A) = -kaP{A) + k^p{C) 
p{B) = -kpp{B) + k^p{A) 
p{C) = -k,p{C) + kpp{B) 

with p{A) -\- p{B) + p{C) — 1. The steady-state probabihties are readily shown to be 
Po(S) = f^Po(^), Po(C) = l^po(A), with 



Po{A) = 



kj^ k'y 



k^k^ ~t~ k^kfy ~\~ kfykj^ 

The rate equations can be integrated straightforwardly, yielding expressions for the time- 
dependent transition probabilities. For example, P{Y\X) is obtained by using the initial 
condition p{Z, t — Qi) — Sz,x and integrating to time t — t. At constant values of the 
reaction rates, the system rapidly evolves towards the steady state. While detailed balance 
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does not generally hold, the relation P{X\Y)p(Y) = P{Y\X)p{X) gets fulfilled as t — )• oo. 
This is illustrated in Fig. [H where we plot the detailed balance ratio p(g|^)p(^j as a function 
of time for fixed values = = 1 and various values of k^. Amusingly, the approach to 
stationarity is non-monotonous for k^ < 4. 




FIG. 1: (Color online) Detailed balance ratio for the three-states cyclic model as a function of time 
for various values of k^, with ka = = 1. Note the non-monotonic approach to stationarity when 
k^ < 4. 

Our focus in this paper is on situations where the reaction rates are not constant but are 
changed in time. The protocol that we use is the same as that discussed in the previous 
Sections: some (or all) reaction rates are changed M + 1 times over a total period of time 
T, such that between two changes the rates are kept constant for a time interval of length 
T = T / (M + 1). The transition probabilities between the different states can be found ana- 
lytically for the simpler cases. For the more complex situations, we obtain these transition 
probabilities by numerically integrating the rate equations. Once we have the transition 
probabilities, we can compute the probability distribution of using Eqs. ([7j) and ([T]). 

As an example, we show in Fig. [2] probability distributions obtained for our three-state 
model where one rate, namely fc^, is increased from fc^ = 1 to fc^ = 4 in M = 15 steps, see 
Fig. [2^. The probability distribution for this forward process is denoted by Pf{4>) and the 
different curves correspond to different total times T. In Fig. [2]d we show the probability 
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distribution Pr{(P) for the reversed process where we start with = 4 and decrease that 
rate to A;-^ = 1 in the same number of steps. 




FIG. 2: (Color online) Probability distributions for the cyclic three-state model for (a) the forward 
process and (b) the reversed process. The different curves correspond to different total times T. In 
the forward process the rate k-y is changed from 1 to 4 in M = 15 steps, with k^ = 0.5 and /c^ = 1. 



Looking at these probability distributions, the first thing one notices is their irregular 
structures for small times T, characterized by pronounced peaks. These peaks become less 
and less prominent when T increases, yielding a smooth distribution in the long-time limit. 
In addition, the distributions rapidly converge to a limiting curve when T increases, and no 
notable changes in the shapes of the distributions are measured when T exceeds 50. 

In order to verify the predictions of the previous Sections, we vary the reaction rates 
in different ways and compute the fluctuation ratio Pf(0)/Pr(— 0). Some of our results 
are summarized in Fig. [3l Whereas in Fig. |3t only the rate k^ is changed, the rates ka 
and kp being constant, in Fig. [3]d we vary all three rates in an independent way. The first 
case can be viewed as a simple example of a system that is equivalent to a two-state coarse- 
grained system (the states A and B can be coarse-grained into a single state), see Section III. 
Consequently, we expect Crooks relation 

P^(0)/P«(-0) = 

to be fulfilled, independently of the length of the time interval. As shown in Fig. |3^, this is 
indeed the case. When all rates are varied independently, the system can not be replaced by 
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an equivalent coarse-grained two-state system, and Crooks relation does not hold, as shown 
in Fig. |3]3 for T = 2. For very large T, however, the process becomes slow, and Crooks 
relation is then again recovered, in accordance with our discussion in Section II. 

2.0 
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0.3 



0.0 - 



-0.3, 



I ' I 



I ' I 



(a) 



T=0.05 
T=50 



I . I 



I . I 



-0.3 0.0 0.3 0.6 0.9 1.2 




-1.2 -0 



0.4 0.0 0.4 0.; 



FIG. 3: (Color online) Fluctuation ratios for the three-state cyclic model. In (a) only the rate 
is changed from /c^ = 1 to A;^ = 4 in M = 15 steps, the other rates being kept fixed, with ka = 0.5 
and ki3 = 1. In (b) all three rates are varied independently: ka from 2.5 to 1.5, kjs from 0.75 to 
2.25, and k^ from 1.1 to 4.1, again in M = 15 steps. For both cases the fluctuation ratios for two 
different times T are shown. 



B. Six states 

In order to fully appreciate the coarse-graining procedure and the generality of our results, 
we study in the following the rather complex six-state system given in Fig. |H Inspection of 
that figure reveals that the processes that lead out of the states A and D depend only on 
the rate a. Similarly, processes out of the states B and F depend only on f3, and out of C 
and E, only on 7. Therefore, if a, 7 and /3 are varied independently, this six-state system 
should be equivalent to a three-state system, where the new states result from the union of 
pairs of the original states: AD = AU D, BF = BU F, CE = CUE. If, in addition, we set 
/3 = 7, for example, we have only two independent variables, and our six-state process should 
be equivalent to a two-state system. Whereas in the former case the detailed fluctuation 
theorem should be broken, in the latter case we should recover Crooks relation even for our 
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FIG. 4: A six-state process that can be coarse-grained into an equivalent three-state process when 
a, 13, and 7 are varied independently. 

six-state system. 

We carefully checked that our original six-state model has indeed the same spectrum and 
probabilities for as the coarse-grained systems, in accordance with the theorem proven in 
Section III. In Fig. |5]we vary the three rates a, /3, and 7 independently. Consequently, the 
six-state model can not be reduced to an equivalent two-state model and the Crooks relation 
is not fulfilled. This is different for the case shown in Fig. Ej where /3 = 7 throughout the 
whole process. The six-state model can then be reduced to a coarse-grained two-state model 
and the Crooks relation is fulfilled. 



V. CONCLUSION 



Understanding the properties of systems far from equilibrium is of great importance in 
many fields in physics. However, the challenges encountered in that endeavor remain extraor- 
dinary. In that context, the recent formulation of various fluctuation theorems, applicable 
either to systems in their steady states or to systems driven out of stationarity, constitutes 
an important development, as they allow a certain characterization and classification of var- 
ious far from equilibrium systems. For that reason it is important to further probe these 
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FIG. 5: (Color online) (a) Probability distribution of (p and (b) fluctuation ratio for the six-state 
model when a is changed from 1 to 3 and P from 1 to 2, whereas 7 = 2 is kept constant. The 
changes are done in M = 8 steps, with T = 5. For this choice of the parameters, the model does 
not have an equivalent two-state model and the Crooks relation is not fulfilled. 

theorems, in order to better understand their range of applicability. 

In this work we have focused on the Hatano-Sasa entropy 0, a quantity which reduces 
to the dissipative work in the special case of a system driven out of equilibrium, and that 
remains well defined even in the absence of microscopic reversibility. This entropy fulfills 
both an integral and a detailed fluctuation theorem for slow systems, i.e., systems which 
reach the steady state after each change in the external parameters. Most importantly, we 
show that for large classes of processes with many states one can find equivalent processes 
with fewer states that have the same spectrum of (f> and the same probabilities for each 
0. In the special case that this coarse-grained system is a two-state system, the original 
many-state process verifies Crooks relation, and this even when detailed balance is broken 
due to the absence of microscopic reversibility. 

Our results show that Crooks detailed fiuctuation theorem is much more widely applicable 
than thought previously, provided that the Hatano-Sasa entropy is used for characterizing 
the system. This opens the possibifity to describe through fiuctuation theorems wide classes 
of systems that have not been considered in that context previously. 
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FIG. 6: (Color online) (a) Probability distribution of 4> and (b) fluctuation ratio for the six-state 
model when a is changed from 1 to 5 whereas /3 = 7 is changed from 2 to 4. The changes are 
done in M = 8 steps, with T = 5. For this choice of the parameters, the model is equivalent to a 
two-state model and the Crooks relation is fulfilled. 
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